Multipath data streaming over multiple wireless networks

ABSTRACT

Embodiments described herein relate to real-time streaming of large quantities of time critical data over multiple distinct networks from a wireless communications device. More specifically, embodiments described herein may address challenges and problems of maintaining consistent data reception quality when faced with the anomalies of a moving sender that is sending data using a relatively unstable radio frequency (RF) method. This may be achieved by converting single source data into multiple data streams, placing them in transport buffers and storing them for forwarding. A plurality radio frequency interface modules provide wireless connectivity to a plurality of wireless networks for transmission over the plurality of networks. Links are maintained to provide feedback on network connections to allow for the transfer of data from one network to another, from one radio frequency interface module to another, and to adjust the amount of data being transmitted.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 12/499,151 filed Jul. 8, 2009, the contents of which are herebyincorporated by reference in its entirety.

FIELD OF THE INVENTION

Embodiments described herein relate to multipath data streaming over oneor more connected wireless networks, and in particular to multipathaudio/video data streaming over one or more connected wireless networksin real time.

BACKGROUND

Delivery of large quantities of continuous real-time data from a mobiletransmitter to a receiver is a complex problem. The receiver may belocated anywhere and connected thereto via one or more connectedwireless networks. The data stream may be compressed and transmittedover one or more wireless links before arriving at a server where theoperation is reversed, and the data stream recreated. Delivery of largequantities of continuous real-time data may be apparent in the field oflive news gathering, for example, where timely delivery of a story tothe studio in a broadcast quality format may be critical for success.

Existing technologies such as satellite or microwave fixed link mayreturn a (live) video stream to a studio, but may have shortcomings. Forexample, satellite or microwave fixed link may take extra time to setup, may be expensive, and once set up may be difficult to be mobile.These solutions may have dedicated channels and a fixed bandwidth thatprevents stream failure; however, sporadic drop-outs due toweather/catastrophic failure are still possible, but may be infrequent.

Cellular data is a possible alternate mechanism to satellite ormicrowave for sending data from a mobile transmitter. Cellular data isgenerally available, available while mobile, and relatively inexpensive.Cellular data may also be relatively low bandwidth, prone totransmission errors and lag due to resource contention, and may besubject to increased interference while moving.

A live audio/video stream has certain characteristics that make itunique, especially when compared with general data (e.g. web browsingdata), stored audio/video data (e.g. content stored on a site such as aYouTube™ or Netflix™).

Stored audio/video solutions typically have the ability to buffer dataat the start of stream playback, or at any time during playback if therequired frames have not yet arrived. Initially, the player storesenough frames from the stream in a buffer to prevent video loss. If thestream is interrupted, the player is able to continue playing from thebuffer, assuming that when the stream is restored, the frames will catchup and the buffer will be refilled. If the buffer empties, the playerstops and displays the ‘buffering’ message while waiting for more data.In some circumstances, particularly where the available bandwidth is notsufficient to maintain the stream the result is a ‘stutter’ effect wherethe output repeatedly stops and starts.

For some industries such as broadcast, stopping to refill a buffer inthe middle of a live audio/video stream may not be acceptable. If thereceiver pauses to refill the buffer, the glass-to-glass latency, asdefined herein, increases.

Professional live broadcast video is real-time with a requirement forfixed glass-to-glass latency. This is required because the producer ofthe live broadcast will typically have the sequence of eventschoreographed, so any change in the glass-to-glass latency would affectthe timing of all other events. A fixed glass-to-glass latency meansthat the response to a video frame that is not delivered on time is toskip it rather than pause the stream to wait for it to arrive. Due tothe way video compression works, the loss of even a single video framemay significantly impact the ability of a system to decompress thestream. This is because the most efficient video compressionimplementations only encode the differences between video frames tominimize the volume of data. As a result, the loss of a single frame ofvideo at the receiver can have significant effects on the reconstructedstream, ranging from persistent video artifacts, black screen, to evencomplete decompression failure until the next Group of Pictures (“GOP”,defined herein) begin.

One solution to the loss of frames may be to choose a glass-to-glasslatency that is large enough to absorb any expected networkinterruptions. For example, a 30 second glass-to-glass latency meansthere would be close to 30 seconds available to retransmit missing orincomplete frames. For situations requiring a shorter glass-to-glasslatency, such as a live interview, this solution may not be feasible.Crucially, in some industries, even delays of seconds can make a videostream unusable or irrelevant. Adding an extended delay to the beginningof a stream, which may still have output errors due to transmissiondelay, is not feasible for these industries.

Transmitting broadcast quality live video in real time over cellularwireless networks is difficult. There are many wireless capable devices,even some using the latest network technologies (such as Long TermEvolution or LTE for example), that are physically capable oftransmitting at the bitrates required for video for short periods.However, the devices may fail when required to reliably transmit a livevideo stream in real time. The failure may be caused by manyuncontrolled variables, such as from physical objects causinginterference, the weather, the contention that occurs when multipledevices share the same limited network resources, and so on.

In the world of large quantity time critical data, some solutions arefocused on delivery of data transmissions to wireless receivers, but notfrom wireless transmitters. Cellular network technology implementationsmay be configured to prioritize delivery of data to a mobile device. Asa result, the down channel (e.g. from the wireless network to the mobiledevice) typically has far more available bandwidth than the up channel(from the mobile device to the wireless network). This may be a worstcase scenario for a mobile device that is generating and attempting totransmit the volume of audio/video data required to send broadcastvideo.

In the field of data communications, multiple data channels may be usedto augment data throughput and solve some problem of deliveringreliable, high quality data transmissions, such as video data. The paper“Distributed Video Streaming with Forward Error Correction”, by ThinhNguyen and Avideh Zakhor proposes one such method. An approach known inthis field includes opening multiple paths and adjusting data ratesbased on the throughput actually reaching the receiver. This approachtypically focuses on extremely specific calculations to maximizeefficiency and Forward Error Correction (FEC) through a control channelto adjust the data rates per channel. These types of solutions generallyfail over wireless network topologies as a result of the many anomaliesthat abound in a wireless medium, which may cause sudden and dramaticchanges in latency and throughput. Moving wireless transmittersexperience problems such as dynamic fading, dead zones, dramatic latencydifferences, echo effects, the ability to receive but not transmit data,RF interference, immediate channel loss, and channel re-allocation tovoice cell phone users.

There exists a need for improved systems and methods for transmitting orstreaming large volume audio/video data transmission (from a mobiledevice) over one or more wireless networks in real time.

There exists a need for systems and methods that address at least someof the challenges and problems associated with maintaining consistentdata reception quality when faced with the anomalies of deliveringaudio/video data stream from a stationary or moving mobile wirelesstransmitter in real time or near real time.

SUMMARY

In one aspect of an embodiment, a system for transmitting large volumesof source data over a plurality of wireless networks is disclosed. Thesystem may comprise a buffer management and transport controllerconfigured to divide an encoded data stream into a plurality of datapackets for buffering into multiple transport buffers; and a pluralityof radio frequency interface modules configured to wirelessly transmitthe plurality of data packets to the plurality of wireless networks toaggregate throughput across the plurality of wireless networks, whereinthe plurality of radio frequency modules are further configured toprovide wireless connectivity conditions to the buffer management andtransport controller regarding the plurality of wireless networks,wherein the plurality of wireless networks are operated by two or morenetwork carriers; and wherein the buffer management and transportcontroller is further configured to process the wireless connectivitycondition to: for a data packet of the plurality of data packets andprior to a transmission of the data packet, withdraw the data packetfrom a transport buffer of the multiple transport buffers and select aradio frequency module for the transmission of the data packet over atleast one of the plurality of wireless networks.

In another aspect, the system may comprise a data transmission controland reassembly module configured to receive the plurality of datapackets from the plurality of wireless networks for reassembly.

In another aspect, the data transmission control and reassembly modulemay be configured to provide receiver feedback to the buffer managementand transport controller regarding additional wireless connectivityconditions for the plurality of wireless networks, and the buffermanagement and transport controller may be further configured to processthe receiver feedback to select the radio frequency module for thetransmission of the data packet.

In yet another aspect, the buffer management and transport controllermay be configured to match an input rate for the plurality of datapackets with a withdraw rate for the plurality of data packets at atransport buffer of the multiple transport buffers using the wirelessconnectivity conditions.

In one aspect, the wireless connectivity condition may comprise one-waynetwork latency, and the one-way network latency may comprise a delaybetween when a data packet is transmitted and when the data packet isreceived.

In another aspect, at least one radio frequency interface modules of theplurality of radio frequency interface modules may concurrently maintainmultiple network connections to the plurality of wireless networks.

In yet another aspect, the transport controller may be furtherconfigured, using the received wireless connectivity conditions, todynamically and selectively redirect a data packet of the plurality ofdata packets from one radio frequency interface module to another radiofrequency interface module of the plurality of radio frequency interfacemodules, and from one available wireless network to another availablewireless network of the plurality of wireless networks, whereby datathroughput to a destination receiver is maintained.

In still another aspect, the source data may be transmitted in real-timewith a constant delay.

In one aspect, the system may comprise a data source configured tocapture a continuous real-time stream of audio-video data for theencoded data stream.

In another aspect, the wireless connectivity conditions may compriseinformation relating to: an error rate for each of the plurality ofradio frequency interface modules based on the assembled packets thatare not delivered at the destination for re-assembly; and a capacitycondition for each of the plurality of radio frequency interface modulesbased on the assembled packets that are delivered at the destination forre-assembly.

In another aspect, the buffer management and transport controller may beconfigured to control: delivery of each data packet of the plurality ofdata packets from a transport buffer of the multiple transport buffersto a corresponding radio frequency interface module of the plurality ofradio frequency interface modules; and transmission, for re-assembly ata destination, of each data packet of the plurality of data packets fromthe plurality of radio frequency interface modules for wirelesstransmission over one of the plurality of wireless networks.

In yet another aspect, the buffer management and transport controllermay be configured to use the received wireless connectivity conditionsto control at least one of the quantity and the quality of the capturedcontinuous real-time stream.

In still another aspect, the buffer management and transport controllermay be further configured to route redundant packets to the plurality ofradio frequency modules for increased transmission reliability.

In one aspect, the wireless connectivity conditions may be selected fromthe group consisting of: one-way network latency, glass-to-glasslatency, transmit collisions at one or more said simultaneouslyavailable different wireless networks; the number of the modules in thewireless radio transmission with each said simultaneously availabledifferent wireless network; throughput requirements of the plurality ofmodules for the wireless radio transmission on each said simultaneouslyavailable different wireless network; coverage drop-off indicators; timebetween successful transmission attempts; status values; Received SignalStrength Indicator (RSSI); packet retransmissions; average packet lossper link; sender bandwidth; destination bandwidth; current availablebandwidth; and combinations thereof.

In another aspect, the buffer management and transport controller may befurther configured, using the wireless connectivity conditions, tocontrol a transmission rate of data packets to a corresponding radiofrequency interface module of the plurality of radio frequency interfacemodules from the multiple transport buffers.

In another aspect, at least one of the plurality of radio frequencyinterface modules may be configured to apply congestion control toestablish and maintain a transmission rate of data packets.

In yet another aspect, the at least one of the plurality of radiofrequency interface modules may be configured to apply congestioncontrol by determining a congestion avoidance threshold.

In still another aspect, an initial value of the congestion avoidancethreshold may be determined as a function of a measured transmissionrate.

In another aspect, a target transmission rate may be determined based onat least one of the congestion avoidance threshold and a measuredtransmission latency.

In one aspect, the congestion avoidance threshold may be increased whena measured transmission latency is within a pre-defined range.

In another aspect, the at least one of the plurality of radio frequencyinterface modules may be configured to transmit data packets at thetarget transmission rate.

In another aspect, the plurality of radio frequency interface modulesmay be configured to transmit the plurality of data packets to theplurality of wireless networks simultaneously or near simultaneously.

In still another aspect, each of the plurality of wireless networks mayhave disparate network characteristics or conditions.

In one aspect of an embodiment, a method for transmitting large volumesof source data over a plurality of wireless networks is disclosed, themethod may comprise: dividing an encoded data stream into a plurality ofdata packets; buffering the plurality of data packets into multipletransport buffers; determining wireless connectivity conditionsregarding the plurality of wireless networks, wherein the plurality ofwireless networks are operated by two or more network carriers;wirelessly transmitting the plurality of data packets to the pluralityof wireless networks to aggregate throughput across the plurality ofwireless networks; and processing the wireless connectivity conditionto: for a data packet of the plurality of data packets and prior to atransmission of the data packet, withdraw the data packet from atransport buffer of the multiple transport buffers and dynamicallyselect a radio frequency module for the transmission of the data packetover at least one of the plurality of wireless networks.

In another aspect, the method may comprise receiving the plurality ofdata packets from the plurality of wireless networks for reassembly.

In another aspect, the method may comprise receiving receiver feedbackregarding additional wireless connectivity conditions for the pluralityof wireless networks, and further processing the receiver feedback toselect the radio frequency module for the transmission of the datapacket.

In yet another aspect, the wireless transmitting may compriseconcurrently maintaining network connections to the plurality ofwireless networks.

In still another aspect, the plurality of data packets may betransmitted such that source data is transmitted in real-time with aconstant delay.

In one aspect, the method may comprise capturing a continuous real-timestream of audio-video data, and encoding the captured stream ofaudio-video data to form the encoded data stream.

In another aspect, said wirelessly transmitting may comprise applyingcongestion control to establish and maintain a transmission rate of datapackets.

BRIEF DESCRIPTION OF DRAWINGS

Various embodiments will now be described, by way of example only, withreference to the following drawings, in which:

FIG. 1 is a block diagram presenting an overview of an exemplary systemfor transmitting audio and video data in real time or near real time;

FIG. 2 illustrates an exemplary embodiment of a multipath systemspanning one or more wireless networks;

FIG. 3 illustrates an exemplary basic queuing model applicable in amultipath system over one or more wireless networks;

FIG. 4 is a block diagram of an exemplary multipath system with multiplewireless networks according to some embodiments;

FIG. 5 is a block diagram of an exemplary system with multiple wirelessnetworks according to some embodiments;

FIGS. 6 a and 6 b are block diagrams of an exemplary system showingdelivery of data packets using feedback mechanisms according to someembodiments;

FIG. 7 is another block diagram of an exemplary system showing deliveryof data packets over multiple wireless networks using feedbackmechanisms;

FIG. 8 is a block diagram of an exemplary system showing additionalfeedback mechanisms;

FIG. 9 is a block diagram of an exemplary system showing simultaneousdelivery over multiple networks to maximize throughput;

FIG. 10 is a flow chart illustrating the flow of data from the mobilesource to the destination;

FIG. 11 is a flow chart illustrating the flow of data to determine whichpaths to use for delivery to a destination; and

FIG. 12 is a flow chart illustrating an exemplary life cycle of avideo/audio frame.

For simplicity and clarity of illustration, where consideredappropriate, reference numerals may be repeated among the figures toindicate corresponding or analogous elements or steps. In addition,numerous specific details are set forth in order to provide a thoroughunderstanding of the exemplary embodiments described herein. However, itwill be understood by those of ordinary skill in the art that theembodiments described herein may be practiced without these specificdetails. In other instances, well-known methods, procedures andcomponents have not been described in detail so as not to obscure theembodiments generally described herein.

DETAILED DESCRIPTION Definitions

The terms used throughout this specification generally have theirordinary meanings in the art, within the context of the presentdisclosure, and in the specific context where each term is used.Further, certain terms are defined in this section, or elsewhere in thespecification, to provide additional guidance to the reader.

Glass-to-glass Latency (G2G Latency) as used herein may refer to thetime delay between the time of capture of a data source (e.g. at theglass of the camera lens) using a data capture device or equipment andthe time of delivery of the reassembled data stream to the intendedreceiver, such as a data transmission control and reassembly module asdescribed herein. In the simplest case, the receiver could be a directlyconnected video output.

Network Latency as used herein may refer to the time from when a packetof data is delivered to a Radio Frequency (RF) interface until that samepacket of data arrives at a receiver after passing through the connectednetwork(s). This is the one-way time period that a single packetrequires to traverse the connected (wireless) network(s), from thetransmitter to the receiver.

Broadcast Quality as used herein refers to a minimum standard of qualitythat the network producers will accept. Video meeting this standard isreal-time (constant delay), has a minimal glass-to-glass latency (<1sec), and no errors in output due to lost frames.

A Connected Network as used herein may refer to network infrastructurewhich communicates with an RF interface on a data capture device (e.g.mobile unit) and acts as a gateway to another network, such as the widerInternet, passing data to a designated receiver. Different carriers mayoperate one or more networks.

A Data Stream as used herein may refer to the continuous delivery ofinformation or data signals, from a data source, to be passed to areceiver. The data source for the data stream may be generated by a datacapture device. In the case of an audio or video data stream, theinformation is typically composed of discrete units known as frames.

A Video Frame may represent a single image in a series of imagesrequired to reconstruct a moving picture.

An Audio Frame may be a discrete segment of a continuous audio datastream. Embodiments described herein may divide a continuous audio datastream into a stream of fixed length segments, each of which mayrepresent a slice of time. Played back in sequence, the audio frames cancomprise the entire audio data stream.

A Video Stream or Video Data Stream may refer to a continuous sequenceof video frames, typically presented at one of the industry standardfrequencies. For instance, the standard NTSC requires frames to bedelivered at 29.97 frames/second. A video stream may or may not includean audio stream. Embodiments described herein may divide a continuousvideo data stream into a stream of fixed length segments, each of whichmay represent a slice of time.

A Group of Pictures or GOP is a term from video compression terminology.A GOP may refer to a set of image frames characterized by a leadreference frame followed by one or more incremental reference frames.Typically, the lead frame (I Frame) is significantly larger than thefollowing reference frames (P Frames or B Frames) since the referenceframes describe only the changes from a previous frame (in the case of aP Frame) or an upcoming frame (in the case of a B Frame).

Real-Time Delivery of Data as used herein may refer to a data streamthat is output from the receiver (after reassembly of received datapackets) at the same rate, and at almost the same instant as it wascaptured from the source. There may not be a true instantaneous outputof data after transport across the one (or more) connected wirelessnetworks, so real time as used herein may refer to delayed real-time ornear real-time. Embodiments described herein may provide a real timesolution that may add a fixed length delay between capture andsubsequent play back. The delay is constant and as short as possible.The delay may be less than one second, though longer may be requireddepending on the connectivity available. This delay may also be referredto as the Glass-to-glass latency, as defined previously.

Buffering as used herein may refer to the process of pausing a receivedstream of audio/video data until the receiver has enough of a reserve ofthe stream to allow for brief pauses in the delivery of content withoutaffecting the output. A typical streaming solution may pause to bufferthe incoming stream at the beginning of play back, and whenever thereserve empties. For delayed real time solutions, buffering may onlyoccur initially, when the stream is first started, any subsequentoccurrences increase the glass-to-glass delay and may cause the outputto fail catastrophically. Embodiments described herein may match therate of data going into the buffer with the rate of data being takenfrom the buffer based on received feedback regarding network conditions.The control of the input rate to the output rate may be determined usingthe received feedback regarding network conditions.

Lost Packets as used herein may refer to data packets that are notdelivered to the receiver in time for reassembly into the audio or videoframe, when that frame is required for play out. In a system withforward error correction, redundant data within the stream of datapackets may allow for a limited number of packets to be lost withoutconsequence. In the case where too many packets are lost, the framecannot be reconstructed and is subsequently lost, causing an error inthe receiver output. There may be no capacity for delay beyond thatbuilt into the Glass-to-glass latency. If a packet is not delivered ontime, it is considered lost and the frame of which it is a component maynot be reconstructed.

Lost Frames as used herein are frames of audio or video data which couldnot be reconstructed in time for output at the receiver as scheduledwithin the fixed G2G latency. Lost Packets, as defined above, may resultin a frame being corrupted and therefore lost. This is not always thecase as lost packets which are detected in time may be recovered. A lostframe can results in an error in the reassembled output stream.

Wireless Connectivity Conditions, Network Conditions, or Feedback asused herein may refer to information received from different modules,components and sources regarding network performance for transmissionand reception of data packets. The feedback may be received fromcomponents involved in the transmission of the data stream andcomponents involved in receiving the transmitted data stream. Feedbackmay also be inferred by observation of components which are not directlyavailable for measurement. Example wireless connectivity or networkconditions include (but are not limited to): network latency (e.g.instantaneous or aggregated latency), packet loss, sender bandwidth ortransmitted bandwidth, receiver bandwidth or received bandwidth, metricsspecific to wireless links (e.g. RSSI, RSCP, EC/IO, CQI), and RFInterface (modem) buffer fullness. Feedback encompasses instantaneousand various statistically filtered versions of the aforementionedvariables, as well as derivative measurements such as rates of changeand statistical correlation between any of these variables.

As used herein, and unless the context dictates otherwise, the term“coupled to” is intended to include both direct coupling (in which twoelements that are coupled to each other contact each other) and indirectcoupling (in which at least one additional element is located betweenthe two elements). Therefore, the terms “coupled to” and “coupled with”are used synonymously.

Throughout the following discussion, numerous references will be maderegarding servers, services, interfaces, portals, platforms, or othersystems formed from computing devices. It should be appreciated that theuse of such terms is deemed to represent one or more computing deviceshaving at least one processor configured to execute softwareinstructions stored on a computer readable tangible, non-transitorymedium. For example, a server can include one or more computersoperating as a web server, database server, or other type of computerserver in a manner to fulfill described roles, responsibilities, orfunctions. One should further appreciate the disclosed computer-basedalgorithms, processes, methods, or other types of instruction sets canbe embodied as a computer program product comprising a non-transitory,tangible computer readable media storing the instructions that cause aprocessor to execute the disclosed steps. One should appreciate that theembodiments described herein may provide improved transfer ofaudio/video data through one or more connected wireless networks in realtime, with a fixed glass-to-glass latency. Embodiments described hereinmay further provide transfer of audio/video data in real time by routingon a data packet basis over multiple network connections (e.g. differentnetworks, networks provided by multiple carriers) simultaneously tomaximize throughput. The routing on a data packet basis may be based onfeedback received from receivers and radio frequency modules. A routingdecision for a given data packet may be made before transmitting thedata packet based on the received feedback.

The following discussion provides many example embodiments of theinventive subject matter. Although each embodiment represents a singlecombination of inventive elements, the inventive subject matter isconsidered to include all possible combinations of the disclosedelements. Thus if one embodiment comprises elements A, B, and C, and asecond embodiment comprises elements B and D, then the inventive subjectmatter is also considered to include other remaining combinations of A,B, C, or D, even if not explicitly disclosed.

The embodiments of the systems and methods described herein may beimplemented in hardware or software, or a combination of both. Theseembodiments may be implemented in computer programs executing onprogrammable computers, each computer including at least one processor,a data storage system (including volatile memory or non-volatile memoryor other data storage elements or a combination thereof), and at leastone communication interface. For example, and without limitation, thevarious programmable computers may be a server, network appliance,set-top box, embedded device, computer expansion module, personalcomputer, laptop, personal data assistant, cellular telephone,smartphone device, UMPC tablets and wireless hypermedia device or anyother computing device capable of being configured to carry out themethods described herein.

Program code is applied to input data to perform the functions describedherein and to generate output information. The output information isapplied to one or more output devices, in known fashion. In someembodiments, the communication interface may be a network communicationinterface. In embodiments in which elements of the invention arecombined, the communication interface may be a software communicationinterface, such as those for inter-process communication. In still otherembodiments, there may be a combination of communication interfacesimplemented as hardware, software, and combination thereof.

Each program may be implemented in a high level procedural or objectoriented programming or scripting language, or both, to communicate witha computer system. However, alternatively the programs may beimplemented in assembly or machine language, if desired. The languagemay be a compiled or interpreted language. Each such computer programmay be stored on a storage media or a device (e.g., ROM, magnetic disk,optical disc), readable by a general or special purpose programmablecomputer, for configuring and operating the computer when the storagemedia or device is read by the computer to perform the proceduresdescribed herein. Embodiments of the system may also be considered to beimplemented as a non-transitory computer-readable storage medium,configured with a computer program, where the storage medium soconfigured causes a computer to operate in a specific and predefinedmanner to perform the functions described herein.

Furthermore, the systems and methods of the described embodiments arecapable of being distributed in a computer program product including aphysical, non-transitory computer readable medium that bears computerusable instructions for one or more processors. The medium may beprovided in various forms, including one or more diskettes, compactdisks, tapes, chips, magnetic and electronic storage media, volatilememory, non-volatile memory and the like. Non-transitorycomputer-readable media may include all computer-readable media, withthe exception being a transitory, propagating signal. The termnon-transitory is not intended to exclude computer readable media suchas primary memory, volatile memory, RAM and so on, where the data storedthereon may only be temporarily stored. The computer useableinstructions may also be in various forms, including compiled andnon-compiled code.

Overview

Transmitting live video in real-time over disparate connected networksis often difficult. Disclosed herein are various embodiments of methodsand systems for the real-time (or near real time) transmission of highvolume, time critical data over wireless networks. The embodiments mayinclude at least one of the following concepts:

-   -   Data streams in real time or near real time. In the case of        (broadcast) audio/video, this may require a guaranteed        glass-to-glass latency traded off with a variable quality.    -   Multiple shared networks operated by multiple carriers. The        design may use a multi-network bonded transport to select the        best path for a given data packet at a given time.    -   Wireless networks may be shared. There may be competition for        shared wireless network resources between other data        applications and voice calls. Connection congestion can be        detected and mitigated.

ADVANTAGEOUS EFFECTS

A real time video stream is extremely sensitive to any delivery delayexperienced by the transmitted data packets. The glass-to-glass latencymust be set large enough to accommodate the expected network latency,otherwise data packets may not arrive in time for reassembly which may,in turn, cause errors and undesirable display effects in the outputstream. The glass-to-glass latency is typically not allowed to vary oncea stream has started transmission.

Embodiments described herein may combine multiple wireless or otherforms of communication channels, which may resolve technical issues withcellular data, while increasing the speed and reliability.

Embodiments described herein relate to the transfer of audio/video datathrough one or more connected wireless networks in real time, with afixed glass-to-glass latency. The wireless networks may be provided orserviced by different network carriers.

Embodiments described herein may further relate to the transfer ofaudio/video data in real time by routing on a data packet basis overmultiple connections simultaneously to maximize throughput.

Embodiments described herein may provide improved transfer ofaudio/video data through one or more connected wireless networks in realtime, with a fixed glass-to-glass latency.

Audio/video data, when represented as a digital data stream, requirelarge amounts of bandwidth to send an acceptable output to thedestination receiver. This bandwidth requirement is capable ofoverwhelming most available wireless communications channels. Further,the challenges increase when faced with the anomalies that arise whenthe transmitter is in motion, or when there are other devices contendingfor the same scarce wireless resources, or other environmentalconditions cause the wireless connection to slow or even fail.

Embodiments described herein may further relate to receiving feedbackfrom system components (e.g. radio frequency modules, receivers), suchas for example, feedback on network latency, to optimize usage andthroughput of one or more connected wireless networks in real time. Thefeedback may be used to make routing decisions before transmitting datapackets.

A data stream may be delivered on a best effort basis, with a consistentdelay between data capture at the source and data play back at thereceiver. There may be no provision for buffering as the output istypically played back in real time. Frames which cannot be recreated intime may be dropped, causing a display error in the output stream. Videoerrors may result in screen distortions or a completely black screen.Both conditions are typically unacceptable to industries which use datacapture and playback equipment.

Embodiments described herein may implement real-time measures to managethe display error rate. As will be described herein, a data stream maybe divided into multiple segments, which may be referred to as datapackets. The data packets may be reassembled at the receiver.

Embodiments described herein may implement real-time measures fortransmitting data streams by routing on a per data packet basis based onfeedback conditions received from transmission related components (e.g.radio frequency modules) and receiver related components.

Embodiments described herein may provide systems and methods to handlecircumstances when the connected network parameters (network latency,error rate, bandwidth, and so on) for one or more networks varyunpredictably, either due to transmitter motion, resource contention orother environmental conditions. The variations in network parameters maybe monitored via feedback received from transmission related components(e.g. radio frequency modules) and receiver related components.

A real time video stream can be extremely sensitive to any variabilityin the delivery delay experienced by the transmitted data packets. Theglass-to-glass latency must be set large enough to accommodate theexpected network latency, otherwise data packets may not arrive in timefor reassembly which may, in turn, cause errors and undesirable displayeffects in the output stream.

After compression, video frames may still be too large to be transmittedover wireless networks without splitting them into smaller pieces(fragments). In the standard OSI network model, fragmentation typicallyoccurs transparently at the network layer (e.g. the IP layer in standardTCP/IP networks). The embodiments described herein seek to avoidtransparent fragmentation at the IP layer in order to increaseefficiency. Transparent fragmentation at the network layer issub-optimal for real-time video applications because if one fragment ofthe original video frame is lost in the network, the fragments that didarrive at the receiver are never delivered to the application, forcingthe sender application to retransmit the entire video frame repeatedlyuntil either all fragments arrive successfully, or the frame expires(i.e. is no longer current from a real-time perspective). In theexemplary implementation, explicit fragmentation is used in that theapplication fragments video frames into data packets before sending themthrough the network. The fragment size is chosen to be slightly smallerthan the standard Internet maximum transmission unit (MTU) size (1500bytes). This size allows data packets to be transmitted across mostconnected networks without transparent fragmentation at the networklayer. More advanced exemplary implementations use well-known Path MTUdiscovery algorithms to discover the maximum MTU between the sender andreceiver, which may be a different value for each wireless connectionavailable to the sender.

In accordance with some embodiments described herein, routing decisionsmay be made on a per data packet basis. For example, one data packetassociated with a data stream may be routed to a first network, andanother data packet associated with the same data stream may be routedto a second (different) network. The networks may be operated bydifferent network carriers. The networks may be the same or differenttypes of networks. The selection of what network to use for transmissionof a given data packet can be made before the data packet is sent. Theselection may be based on feedback received from transmission relatedcomponents (e.g. radio frequency modules) and receiver relatedcomponents.

Embodiments described herein deliver data packets across differentwireless networks, which may be different than delivering data packetsover different network paths within the same network. For example,embodiments described herein transmit packets over different networks toa set of receivers that accept transfer of packets, while operable tochange the rate of packet delivery (based on feedback) selectively fromcarrier by carrier basis, network by network basis, receiver by receiverbasis, or packet by packet basis. Embodiments described herein provideflexible, adaptive, and dynamic tools for data packet transmission.

Embodiments described herein may not require a listing of active basestations for one or more networks, as this may require integration intoa particular network and may not be feasible if a network carrier doesnot cooperate to provide this information. Instead, embodimentsdescribed herein transmit different packets over different networks,which may operate separately and by different carriers. Further,different base stations may operate at different frequencies and whichmake communication between different base stations within the samenetwork and across different networks difficult.

In one exemplary embodiment, a process of capturing video for broadcastis illustrated in FIG. 1. First the video is captured by a camera orsimilar device 302. The resulting video stream is passed to a real timeaudio/video encoder 310 where it is compressed and optionally encrypted.The resulting compressed stream is then directed, still in real time,across a communication network 320 to the intended receiver. Thecommunication network may be as simple as a wire between two devices, afixed/dedicated link, an internal network, the Internet, a fibernetwork, or a cellular communication network. The intended receiver, forexample a real time audio-video Decoder 330, decompresses and decrypts(if required) the video stream. In another embodiment, the intendedreceiver can also be a data transmission control and reassembly moduleas described elsewhere in this disclosure. The reconstructed stream isnow available for distribution or viewing on a connected monitor 350. Animportant consideration is the time T 340 required for the capturedvideo to be encoded, transported, and decoded—this is an example of theGlass-to-glass latency as defined above.

Referring now to FIG. 2, there are further considerations that can bemade when the transmission of encoded data stream is done over one ormore (wireless) networks 320, 322, 324 to a Decoder 330, as shown in thenetwork system 300. For example, after being encoded and optionallyencrypted, the video/audio data stream may be transmitted via multipleradio frequency (RF) interfaces 360, 362, 364 over multiple differentwireless networks 320, 322, 324, respectively. Each network can have itsown characteristics and conditions, including available bandwidth,network latency, glass-to-glass latency, transmission rate, data packetqueue length, input rate, output rate, and so on. So, each network mayhave disparate characteristics and conditions. Consequently, eachnetwork should be treated differently. For example, each network 320,322, 324 may have an independent bandwidth Bn 342, 344, 346, and anetwork latency Ln 352, 354, 356. An embodiment may take into accountthese disparate characteristics to maximize overall throughput whilemaintaining a constant time T 340.

In some embodiments, all connected networks can be assumed to be ashared resource with access and availability shared with the manyconnected devices operated by the general public. The mobile device isoften unable to control access to the network or define a minimumquality of service. As a result, in one embodiment, variability in eachof bandwidth Bn 342, 344, 346, and network latency Ln 352, 354, 356 canbe accommodated by, for example, adjusting the quality of the outputvideo stream. Even on communication networks with a defined quality ofservice, there may exist the possibility of service interruption ordelay that the Encoder 310 should account for.

FIG. 3 depicts an exemplary embodiment of one wireless network in FIG.2. Each communication (wireless) network can have at least two networkcharacteristics: a queue 325 containing data packets received by thenetwork and not yet passed to the Internet gateway, and a rate orbandwidth B1 342—the maximum rate at which the wireless network 320 isable to send data to an Internet gateway.

In another embodiment, one mobile device can continuously transmitsource data through wireless network 320. If the rate at which themobile device sends data packets to the network is greater than theoutput rate B1 342, then the wireless network can temporarily storepackets in the queue 325. As the queue 325 is filled, the measuredlatency L1 352 increases. The behaviour of the queue can vary dependingon the configuration of the wireless network. In some networks, thequeue has a limited size or depth. In other networks, the queue can betreated as having infinite depth, and the network latency may growwithout bounds. Other networks may have a queue per device, or may havea queue of extremely limited length (resulting in frequently droppeddata packets).

On a public network, there are typically multiple devices submittingdata for transmission, all sharing the same output rate B1 342. As aresult, it is impossible for a mobile device to know exactly how fullthe queue is or the exact rate or bandwidth available to the mobiledevice for sending data to the network. Furthermore, the details of theoperation and/or state of a communication or wireless network are oftennot available to the mobile device. An exemplary embodiment may performstatistical analysis on the available feedback (transmit rate, receiverate, instant network latency, averaged network latency, and otherparameters) in order to determine the optimal transmission rate or radiofrequency for any given interval.

In one embodiment, each RF interface can independently implement acongestion control algorithm to establish and maintain a predictable oran appropriate data transmission rate. The algorithm continuouslyupdates a long term, slow moving estimate for the channel's availablebit rate B1 342. This estimate may be referred to as a congestionavoidance threshold (CAThresh).

Traditional congestion control algorithms such as the one used by TCP,follow a pattern known as additive increase, multiplicative decrease(AIMD). When a congestion event occurs, a target transmission rate maybe quickly (or multiplicatively) reduced. The target transmission ratemay also be known as a target bits per second (Target BPS). When thecongestion event disappears, the Target BPS is slowly (or additively)increased. As described herein, in one embodiment, a novel congestioncontrol algorithm comprises at least two differences from thetraditional methods: 1) instead of arbitrary multiplicative decreases ofa target transmission rate (or a Target BPS) in response to congestion,decreases are kept as a function of bits per second as reported by thereceiver (Received BPS); and 2) multiplicative or quick increases ofTarget BPS are allowed if the current Target BPS is below CAThresh (thecurrent estimate for B1 200). The receiver here may be a decoder or adata transmission control and reassembly module.

In one embodiment, an initial value for CAThresh can be determined byincreasing the transmitted instantaneous bits per second (Target BPS)until there is a definitive positive correlation between the increasingtransmission rate (Target BPS) and the measured network latency L1 352.Correlation indicates that the queue 325 is being filled, indicating thechannel's available bit or bandwidth rate B1 342 has been reached orsurpassed. Once this correlation is determined/observed, the initialvalue of the congestion avoidance threshold can be determined as afunction of the measured data transmission rate (Received BPS) reportedby the receiver, such as a data transmission control and reassemblymodule. The function ensures the initial value of CAThresh is below B1342 (which is at least the value of Received BPS).

Immediately after the initial value for CAThresh is determined, theTarget BPS is set to a value slightly below CAThresh, allowing queue 325to be drained, which reduces the measured network latency L1 352. Oncethe latency has stabilized, Target BPS is increased multiplicatively aslong as it remains below CAThresh and latency L1 352 does not increase,meaning queue 325 is not being filled. In addition, if latency starts toincrease while the target bit rate BPS is below CAThresh, the Target BPScan be decreased by a function of Received BPS and reduce the CAThreshaccordingly.

When the target bit rate BPS is equal to CAThresh, both are increasedslowly (additively) as long as the network latency remains in anacceptable (pre-defined) range (and is not positively correlated withthe increased bit rate). The algorithm is probing for more availablebandwidth, in case the available bit rate B1 342 has increased. Once apositive correlation is detected, both parameters are again reduced to avalue that is a function of Received BPS.

There is an exponential time delay between failed additive increaseattempts to prevent repeated/periodic behaviour in the case whereCAThresh already equals B1 342.

Increases in network latency L1 352 seen while the channel's Target BPSis less than or equal to CAThresh are assumed to be transient. If thelatency increases beyond an acceptable range, the Target BPS isimmediately reduced to a value that is a function of Received BPS, andCAThresh is slightly reduced.

If the network latency L1 352 returns to an acceptable level, the TargetBPS is allowed to increase quickly (multiplicatively) as long as it isless than CAThresh. This allows the Target BPS to recover quickly towardthe channel capacity B1 342 in the case where the latency increase was atransient spike. Once Target BPS reaches CAThresh, the algorithm resumesperiodically probing for additional bandwidth.

The congestion control algorithm can also detect when the wirelesscommunication network reduces the available bandwidth B1 342 to a valuebelow CAThresh. If B1 342 has been reduced, each time Target BPS isgreater than B1 342, there is a corresponding increase in the networklatency L1 352. This triggers an immediate reduction of Target BPS to avalue that is a function of the Received BPS, and slightly decreasesCAThresh. This process repeats until CAThresh is less than or equal tothe new value of B1 342 and the network latency stabilizes. Thealgorithm then resumes periodic probing for additional bandwidth.

FIG. 4 is a block diagram of an exemplary system 10, in one embodimentof the invention, with different wireless networks 70, 72, 74 operatedby different carriers. Wireless Networks 70, 72, 74 could include one ormore similar wireless network types, or they could all be unique. Thewireless networks may be associated with different data transmissionrates, and other connectivity conditions. The conditions may bemonitored by RF interface modules 52, 54, 56, 58 and data transmissionand control re-assembly 46 and used by system 10 as feedback.

Wireless networks 70, 72, 74 may be configured differently acrossdifferent carriers, and between one geography and another. Embodimentsdescribed herein may implement routing techniques for data packets ofdata streams for transmission over different network configurations andcarriers. For example, different networks may have different queuingmodels and embodiments described herein may be used for differentqueuing models and without access to detailed information about theunderlying queuing models. Access to such information may be difficultand may require cooperation by carrier.

An exemplary use is in environments where there are large quantities ofdata that must be delivered in real time with a need for mobility and anon-fixed origin or source for the data. An example of this is in thenews and broadcasting industry where information may be captured andimmediately transmitted to a central studio for broadcast. A commonsituation includes live action news coverage cameras that are currentlyconnected to satellite trucks costing hundreds of thousands of dollarsto build and outfit. These costs are prohibitive for smaller newscenters. A cellular-based method is ideally suited for delivering suchlive news content.

In the exemplary system 10, the Data Source 20 exists for capturing datasuch as video feeds, audio/video camera feed, multimedia data, andlogging. When Data Source 20 is video and audio based it may includenormal, high, or extremely high definition audio and video content. Whenthe Data Source 20 is multimedia it may include audio, video, pictures,commands, voice data, television signals and other advanced digitalcontent. Depending on the type of Data Source 20, the data stream 22 mayuse a coupling technique based on the Data Source 20 type. For example,when the Data Source 20 is audio and video content an analog and digitalcamera may be used and may follow standard connection methods such asSerial Digital Interface (SDI), composite video interface or firewire(standard IEEE 1394) for example. Although only one data source 20 isshown for simplicity, system 10 may involve multiple data sources 20.

A data stream 22 from Data Source 20 may be directed to a Data Processorand Encoder 24. In an exemplary environment the Data Processor andEncoder 24 contains one or more high power computer processors andadditional graphic processing chips for high-speed operation. Dependingon the type of data stream 22, content several advanced encoding methodscould be used on the data. The encoding methods could includecompression techniques, encryption techniques and multiple descriptiveencoding (MDC) techniques to name just a few. When the data stream 22 isan audio and video data stream the Data Processor and Encoder 24 encodesthe audio and video stream 22 using a standard compression encodingmethod such as MPEG-4 or the like. For other types of data othercompression, encoding or encryption may be used on the data. Thecompressed and encoded data stream(s) are then delivered to the receiverwhere the Data Transmission Control and Re-assembly module 46reconstructs, decompresses and decodes the data stream into its originalform.

Audio and video may have different characteristics and as a result maybe encoded separately, with different compression mechanisms, eachtailored to the data stream type. The encoding process chosen forcompressing the video sub-stream may also result in additional streams,for example multiple descriptive coding (MDC). Within the MDC splittingmethod there are many different algorithms that could be used to achievethe splitting of a single data stream 22 into Multiple Data Streams 12,one such algorithm is called Polyphase Down Sampling. Other types ofdata splitting algorithms could be used to achieve similar results. Anexemplary implementation may use H.264 to encode and compress the videostream into a single data stream. An audio encoder (e.g., mp3)compresses the audio stream.

Referring also to FIG. 12 there is shown a flow chart of a method 900illustrating the life cycle of a video frame.

The Data Processor and Encoder 24 handles the source audio/video datastream in several steps. As noted, the audio/video data stream may be alive, continuous, real-time data stream.

At 902, the data stream may be split into separate audio and videostreams. Each stream may be composed of multiple frames. The videostream contains sequential individual images and the audio containssegments of continuous sound.

At 904, each frame is time stamped with a time associated with thetransmitter (e.g. transmitter's local time) and sequence numbered forlater ordering and playback.

At 906, the video frames may be compressed and encoded with a variablerate compressor, in the case of the exemplary solution, H.264. Othercompression and encoding techniques may also be used.

At 908, the audio frames may be compressed using a variable ratecompressor such as Opus or a fixed rate compressor such as MP3 orsimilar technology. Some example implementations may use a slightly lessefficient MP3 encoding mechanism. The resulting output is stateless asnone of the previous or following frames are required to decode thecurrent frame (as is the case for a compression technology such as AAC).This allows the receiver to lose the occasional audio data packet with aminimal impact on the decoded output. Step 906 and 908 may beinterchangeable in order.

At 910, the compressed frames of both streams are passed to the BufferManagement and Transport Controller 28. Although the Buffer Managementand Transport Controller 28 is referred to as one component herein itmay comprise separate, distinct components.

As shown in FIG. 4, multiple data streams 12 may be fed into the BufferManagement and Transport Controller 28 over a link 26. This link 26could be a direct link, for example over a computer Bus, a shared memoryor other fast internal method or over an external method such asfirewire link, a Universal Serial Bus (USB) link, a serial connection,Bluetooth or WiFi wireless link or some other high speed link. In afully integrated system the Data Processor and Encoder 24 could co-existwith the Buffer Management and Transport Controller 28 in the samephysical housing.

The Buffer Management and Transport Controller 28 acts as the maininterface to one or more Radio Frequency (RF) interface modules 52, 54,56, 58 (e.g. RF module). These are coupled by feature 30 to the BufferManagement and Transport Controller 28 using a variety of couplingmethods such as Universal Serial Bus (USB), serial link, Personal MemoryCard International Association (PCMCIA) or direct motherboard busconnection. In FIG. 4 the RF Interface Modules are illustrated as RFmodules 52, 54, 56, 58. As shown in FIGS. 4 and 5, multiple RF interfacemodules 52, 54, 56, 58 may service one wireless network 70. As shown inFIG. 3, a single RF interface module 52, 54, 56, 58 may service multiplewireless networks 70. One or more wireless networks 70, 72, 74 may beoperated by different network carriers, as shown in FIG. 4 (or in someembodiments in FIG. 5). One or more wireless networks 70, 72, 74 may beoperated by the same network carriers, such as may be shown in FIG. 5.

Embodiments described herein may use multiple RF interface modules,wherein the RF interface modules 52, 54, 56, 58 may be assigned to morethan one network 70, 72, 74. The networks 70, 72, 74 may be associatedwith its own network provider or carrier.

Some of the functions of the Buffer Management and Transport Controller28 include receiving the encoded Data Streams (DS) 14 and buffering theminto Multiple Transport Buffers 16 based on complex logic and ensuringthat all Transport Buffers (TB) 34, 36, 39 and 40 make it through the RFInterface Modules 52, 54, 56, 58, over Wireless Networks 70, 72, 74 tothe Data Transmission Control and Re-assembly module or component 46.Buffer Management and Transport Controller 28 may match the rate of datagoing into the buffer with the rate of data being taken from the bufferbased on received feedback, such as Received BPS. Buffer Management andTransport Controller 28 may control the input rate and the output rateusing the received feedback regarding network conditions. Data packetsmay be buffered in transport buffers (e.g. as a long stream of datapackets, or a set of data packets) and selectively and dynamicallywithdrawn from transport buffers and delivered to a selected RFinterface module (associated with one or more networks, and one or morenetwork carriers). The data packets may be continuously andsimultaneously transmitted across the RF interface module and networkfor reassembly at the destination (e.g. receiver).

Multiple wireless networks may be used for simultaneous wirelesstransmissions. This may increase the data throughput of the system. Eachwireless network is associated with an individual maximum throughput.Use of multiple wireless networks simultaneously enables system 10 toaggregate the individual maximum throughputs to provide an aggregatedmaximum throughput. For example, a data stream 14 may be segmented intomultiple data packets (and buffered in transport buffers 34, 36, 38, 40)for provision to RF interface modules 52, 54, 56, 58. Data packets froma single data stream 14 may be transmitted simultaneously over differentWireless Networks 70, 72, 74 to provide an aggregated throughput fortransmission of the data stream 14. The selection of which RF interfacemodule 52, 54, 56, 58 to use for the transmission of a given data packetis made on a per data packet basis with reference to feedback receivedfrom various components of system 10, including RF interface modules 52,54, 56, 58, transmission control and data re-assembly 46, and so on.

A variety of methods and techniques may be used for exchanging RFsignals through Wireless Networks 70, 72, 74. On the server side of theWireless Network 70, 72, 74 land-based methods 44 for connecting toWireless Networks 70, 72, 74 may be used. These connection methodstypically may be TCP/IP based, but it is also well known to use IP aloneand to use connectionless methods such as UDP over IP to create fasterdatagram-based delivery methods. Connection methods 44 may also includeX.25 packet switched networks and asynchronous dial-up lines. Mostadvanced data delivery systems use UDP/IP methods over wireless andcreate their own transport hand-shaking methods to better customize thedelivery of information in the harsh wireless environment. Usingunmodified TCP over wireless networks usually perform poorly overwireless, since random wireless loss is interpreted as congestion.Several optimized TCP protocols have been created to make TCP moreforgiving and better adapted to wireless, but the reliable and in-ordernature of TCP is still not well suited for real-time applications. Asnoted, data packets in accordance with embodiments described herein maysegment UDP/IP packets.

Referring back to FIG. 12, the Buffer Management and TransportController 28, at 912, is operable to divide the video or audio framesinto one or more data packets and place them in the transport buffer(s)16 until an RF interface module 32 is ready and available to send them.An audio frame may fit within a single data packet. Video frames,depending on the state of the encoder and compression rate varysignificantly in size requiring few as one data packet, to many. A datapacket may be different from an IP packet, in that an IP packet may begenerally larger in size than a radio packet. The Buffer Management andTransport Controller 28 may fragment IP packets into smaller datapackets.

The Buffer Management and Transport Controller 28 is operable to segmentthe frames into data packets to enable routing decisions to be made on aper data packet basis (as opposed to a per frame or data stream basis,for example). Making routing decisions on a per data packet basis maydistribute the transmission of a frame or a data stream across multipledifferent networks simultaneously (or near simultaneously) to leverageaggregate throughput of the multiple different networks. That is,embodiments described herein may aggregate data throughput acrossmultiple networks.

One or more transport buffers 16 may be used to deliver frames on a oneto one basis (one transport buffer 16 for each RF interface module 32)or a one to many basis (one transport buffer 16 services one or more RFinterface modules 32). For simplicity, the exemplary implementationshown in FIG. 4 uses a single transport buffer 16 as an interface to allof the connected RF interface modules 32. Buffer Management andTransport Controller 28 may match the rate of data going into thetransport buffers 16 with the rate of data being taken from thetransport buffers 16 based on received feedback. Buffer Management andTransport Controller 28 may control the input rate and the output rateusing the received feedback regarding network conditions. BufferManagement and Transport Controller 28 may analyze receiver andtransmitter related network conditions across multiple networks. BufferManagement and Transport Controller 28 may establish an input rate andan output rate responsive to the input rate, and the transmitter sidenetwork conditions and the receiver side network conditions (e.g.feedback).

At 914, when an RF interface module 32 is ready for data, the next datapacket is selected from the designated transport buffer from one or moretransport buffers 16 delivered to the RF interface module 32 fortransmission over the wireless network 70, 72, 74. That is, the BufferManagement and Transport Controller 28 may make routing decisions (e.g.selection of an RF interface module 32 to use for transmission) on a perdata packet basis. The Buffer Management and Transport Controller 28 maymake routing decisions based on received feedback.

In accordance with some embodiments, the Buffer Management and TransportController 28 may route one or more data packets associated with a datastream to a first RF interface module 32 (and onto a first network 70,72, 74). The Buffer Management and Transport Controller 28 may routeanother data packet associated with the same data stream to a second(different) RF interface module 32 (and onto a different network 70, 72,74). The networks 70, 72, 74 used for transmission of the data packetsmay be operated by different network carriers (FIG. 4). The networks maybe the same or different types of networks. The Buffer Management andTransport Controller 28 may select what RF interface module 32 andnetwork 70, 72, 74 to use for transmission of a given data packet beforethe data packet is sent. The selection may be based on feedback receivedfrom transmission related components (e.g. radio frequency modules) andreceiver related components.

As with all wireless transmissions, general failures occur frequentlyfor a variety of reasons. In an exemplary environment the dividedTransport Buffers (TP) 34, 36, 38 and 40 allow the Buffer Manager andTransport Controller 28 the ability to make use of many connected RFInterface Modules 32. Each of these RF Interface Modules 32independently monitors and reports (complex) wireless network 70, 72, 74connectivity conditions, or feedback. The connectivity conditions (e.g.feedback) may be used by the Buffer Manager and Transport Controller 28to control routing of data packets between different RF InterfaceModules 32.

In another exemplary environment, a single Transport Buffer 34, 36, 38and 40 acts as the interface between the Buffer Management and TransportController 28 and the many connected RF interface Modules 32. RFinterface modules 32 may also maintain an internal Transport Buffer 34,36, 38 and 40 for ease of tracking transmitted and pending data packets.

At 916, the data packet is received over the wireless network 42 attransmission control and data re-assembly 46 for re-assembly into framesof data streams.

FIG. 5 is a block diagram of an exemplary system 50 with multiplewireless networks 70, 72 and 74. In this embodiment the Data Processorand Encoder 24 act to encode and divide data transmission from a DataSource 20. The Buffer Management and Transport Controller 28 has furtherextensions to support a wide range of RF Interface Modules 52, 54, 56and 58. In this environment the RF Interface Modules 52, 54, 56 and 58are located across a plurality of Wireless Networks 70, 72 and 74 whichmay be operated by one or more wireless carriers. These WirelessNetworks 70, 72 and 74 could include one or more similar wirelessnetwork types, such as two GSM EDGE networks, or they could all beunique. In this example illustration, wireless networks 70, 72, 74 use aplurality of base stations 60, 62, 66, 68 to resolve coverage andcapacity issues. The wireless networks may be associated with differentdata transmission rates, and other connectivity conditions.

The system 50 may be used for delivering large volumes of source datainformation simultaneously over one or more wireless networks 70, 72 and74. As noted above, the data processor and encoder 24 may convert acontinuous audio-video data stream into frames. The data processor andencoder 24 may then encode the frames into data packets. System 50 isoperable to segment data streams and frames into smaller data packetsand to distribute transmission of the smaller data packets across themultiple wireless networks 70, 72 and 74.

The RF modules 52, 56, 58 enable simultaneous transmission of datapackets to multiple wireless networks 70, 72 and 74. This may enablerealization of an aggregated maximum data throughput by aggregatingindividual maximum data throughputs associated with each of thedifferent wireless networks 70, 72 and 74.

The Buffer Management and Transport Controller 28 buffers each of theencoded data packets in one or more transport buffers for delivery tothe RF modules 52, 56, 58. The Buffer Management and TransportController 28 selects an RF module 52, 56, 58 on a per data packet basisbased on received wireless connectivity conditions for each of themodules 52, 56, 58 (e.g. feedback). The wireless connectivity conditionsor feedback may also be received from data transmission and controlre-assembly, as described herein. The per data packet selection enablessimultaneous transmission over multiple wireless networks 70, 72, 74 tomaximize throughput. A single data stream is converted into individualdata packets where transmission may be spread across multiple wirelessnetworks 70, 72, 74 simultaneously based on an optimal routing via RFmodule 52, 56, 58 and associated wireless connectivity conditions. Anexample wireless connectivity condition is network latency as definedpreviously.

The Buffer Management and Transport Controller 28 monitors the receivedwireless connectivity conditions (e.g. feedback) to direct each encodeddata packet on a per-packet basis to one of the modules 52, 56, 58. Inan example, one data packet may be routed to a first module 52, 56, 58while another data packet may be routed to a different module 52, 56,58. Similarly, one data packet may be routed to one wireless network 70,72, 74 while another data packet may be routed to another wirelessnetwork 70, 72, 74. This enables simultaneous transmission of a singlecontinuous data stream over the plurality of wireless networks 70, 72,74. As noted, the routing is on a per-data packet basis, a per-RFinterface module basis and a per-network basis to use multiple networks70, 72, 74 simultaneously for a single data stream. An RF module 52, 56,58 maintain multiple network connections to the wireless networks 70,72, 74 simultaneously.

The Buffer Management and Transport Controller 28 is operable, using thereceived wireless connectivity conditions, to dynamically andselectively redirect the data packets from one RF module 52, 56, 58 toanother module 52, 56, 58, if for example, one RF module 52, 56, 58 isnear capacity, experiencing congestion, experiencing errors or delay,and so on. The Buffer Management and Transport Controller 28 isoperable, using the received wireless connectivity conditions, todynamically and selectively redirect the data packets for a data streamfrom one available wireless network 70, 72, 74 to another availabledifferent wireless network 70, 72, 74, whereby data throughput to thedestination receiver (e.g. data transmission control and re-assemblymodule) is maintained.

The system 50 includes a data source 20 operable to capture a continuousreal-time stream of audio-video data. The simultaneous (ornear-simultaneous) transmission may occur in delayed real-time asdefined previously.

In an environment where there are multiple connected RF interfacemodules 32, 52, 54, 56, 58 (either to separate and distinct networks 70,72, 74, or using the same or different carrier), the alternatives fordelivering Data Packets are enhanced. If one of the connected RFinterfaces modules 32, 52, 54, 56, 58 reports congestion or transmissionissues, there are many alternatives to transmit around the slow RFinterface 32, 52, 54, 56, 58. For example, RF interface 32, 52, 54, 56,58 might become busy and start reporting increasing amounts of networklatency. As a result, any data packets assigned for delivery to that RFinterface 32, 52, 54, 56, 58 might slow down dramatically while waitingfor wireless spectrum to become available.

The Buffer Management and Transport Controller 28 is operable to monitorreceived connectivity conditions (e.g. feedback) to detect congestionand transmission issues. Data Packets originally planned for one RFInterface Module 54 might be re-routed by the Buffer Management andTransport Controller 28 to another RF Interface Module 56 and sentthrough base station 66 onto another Wireless Network 72 instead.Examples are shown in the flowcharts of FIGS. 10 and 11. Through themonitoring of feedback or connectivity conditions, the management of RFInterface Modules 52, 54, 56 and 58 helps to provide the BufferManagement and Transport Controller 28 its unique ability to keep highthroughput when delivering data packets over multiple wireless networks70, 72, 74.

In cases where a network 70, 72, 74 is connected to more than one RFinterface modules 52, 54, 56, the increase in available aggregatedthroughput may not as great as it would be if the networks 70, 72, 74were separate (e.g. only connected to one RF interface module). However,there is still an advantage due to the increase in redundancy andincrease of options for RF interface modules. It may be in most cases aprocess of diminishing returns.

In some examples, a single transmitter may have devices or RF interfacemodules for each of the (usually several) different networks 70, 72, 74visible or available. In some circumstances, when there is not enoughnetwork variety, a transmitter may be configured with duplicateinterface modules to the same network. Duplicating wireless networks 70,72, 74 may not necessarily lead to doubling throughput due to networklimitations, but it still may increase available data throughput.

A role of the Buffer Management and Transport Controller 28 is theability to manage decisions around utilizing all attached RF InterfaceModules 52, 54, 56 and 58 for transmission of data packets basedreceived feedback or connectivity conditions. The decisions may relateto which RF Interface Modules 52, 54, 56 and 58 to use to transmit ordeliver a particular data packet. Decisions may be made on a per packetbasis in order to spread the transmission of data packets from a singledata stream over multiple RF interface modules 52, 54, 56 and 58 andnetworks 70, 72, 74.

Delivery to the Data Transmission Control and Re-assemble module 46 mayproceed more smoothly with a greater number of RF Interface Modules 52,54, 56 and 58. In this embodiment the Data Transmission Control andRe-assembly module 46 is particularly configured to connect to eachsupported Wireless Network 70, 72 and 74 and reassemble data packetsfrom all (e.g. three) networks. These links 44 will deliver data packetsfrom each of the RF Interface Modules 52, 54, 56 and 58 which will bere-assembled in order based on the time stamps and sequence numbersplaced into each data packet by the Buffer Management and TransportController 28. Accordingly, a data packet may contain timing andsequence data for use by Data Transmission Control and Re-assemblemodule 46 to re-assemble the data stream.

The encoded Data Streams 22 consist of a series compressed (and possiblyencrypted) video and audio frames. The Buffer Management and TransportController 28 may monitor the reported network feedback (e.g.connectivity conditions), calculate the health of the individual RFinterface modules 52, 54, 56, 58 or networks 70, 72, 74, and theaggregate session of all the RF interfaces or networks combined, andthen act on the results by making routing decisions.

For instance, if the network latency of a connection is slowlyincreasing then the Buffer Management and Transport Controller 28 mayelect to decrease the rate of data packets sent to the associated RFinterface for delivery and redirect data packets to another RF interfacemodule.

The system is operable to evaluate the sum of all the combinedconnection throughput rates and based on that information increase ordecrease the compression and encoding rate (and the resulting actualdata bit rate).

A simplified illustrative system with two connections in a steady state,with both connections transmitting roughly the same amount of data willbe described herein as an illustrative and non-limiting example. This isan example for illustrative purposes and other variations,configurations and modifications may be used for systems.

Network 1 associated with connection 1 may start to experience heavyloading due to increased activity from other mobile devices competingfor a shared wireless spectrum.

As the loading on network 1 increases, the measured network latency mayincrease, or the number of lost packets may increase, or some othernetwork measurement may indicate congestion. This measure may beprovided or communicated as feedback or connectivity conditions. Whenone of these conditions passes a threshold, the Buffer Management andTransport Controller 28 may start limiting the amount of data packetshanded to corresponding RF interface 1 for delivery to network 1.Accordingly, the Buffer Management and Transport Controller 28 may beconfigured with rules for threshold conditions to govern routing of datapackets. The threshold conditions may be tailored or configured tospecific networks and interface modules or types thereof, or may bestandard threshold conditions applicable to all types of networks andinterface modules.

The Buffer Management and Transport Controller 28 may also take steps tostart decreasing the amount of frame data generated by the data source20.

The Buffer Management and Transport Controller 28 may also take intoaccount the delay between the instant a data source decrease isrequested and when the decrease takes effect. During this interval, theBuffer Management and Transport Controller 28 may redirect data packetsto RF interface 2 such that RF interface 2 may temporarily increase theamount of data packets it transmits. The level of padding information onRF interface 2 may be temporarily decreased to accommodate theadditional volume of information.

Reducing the traffic sent to connection 1 allows the wireless network 1to recover and reduce the measured latency. If the latency decreasespast the set threshold, the Buffer Management and Transport Controller28 may be able to once again increase the data packets sent to thatconnection (e.g. network 1 and RF interface 1). However, if the networklatency continues to increase, the Buffer Management and TransportController 28 may continue to decrease the traffic sent to connection 1.If the latency passes the point at which data can no longer be sent witha reasonable expectation of delivery and reassembly in time for playout, then the connection 1 may be removed from the active transmissionpool entirely and used only for retransmission and testing. Theconnection 1 may not be used again until the measured network latencyonce again decreases to a usable rate. This rate may be configured as arule for Buffer Management and Transport Controller 28.

RF interface modules may be limited to a percentage (e.g. 80%) of theircalculated/measured individual maximum bandwidth or throughput. Theremaining space may be reserved for padding, which may refer to extraspace to allow the transmission of backlog and/or lost data packetresend requests. Padding may also be employed to keep the connectiondata rates relatively constant. This may prevent cellular or otherwireless networks from reducing the bandwidth assigned to the RFinterface when the data rate is reduced. Increasing the data rate againlater may not be possible if the network resources have been allocatedto other users, or may come only after a noticeable delay. Accordingly,the threshold values for RF interface modules may factor in padding toreserve extra space or capacity. The threshold value may refer to acapacity condition or network capacity condition that is measured byeither the transmitter or receiver. If measured by the receiver, thefeedback is sent to the transmitter and used in making routing decisionsfor subsequent data packets. Embodiments described herein includemeasured conditions such as one-way network latency, jitter of suchlatency, packet loss, received packet order, and sent and received bitrates.

Referring back to FIG. 3, in one embodiment, the receiver collects rawnetwork and packet delivery information as it is generated by thereceived data stream(s). The collected information is then statisticallyfiltered over a series of intervals, the length of which is determinedby a slow moving average of the network latency L1 352. The Network canbe modelled as a FIFO of “L1 352” ms, so the statistics are calculatedover one complete “filling” of the FIFO.

At the end of every window, the following statistics or values can bereported back to the transmitter:

-   -   The size of the window (in milliseconds);    -   Mean latency of all packets received in the window;    -   Latency standard deviation for all packets received in the        window;    -   Missing packets—packets that were missing from the previous        window, not the current window. In some embodiments it can be        difficult to tell the difference between a late packet and a        missing packet, i.e. a missing packet might just be really late,        so a decision may be made after the subsequent window;    -   Received packets;    -   Received bytes;    -   Latency jitter—A filtered average of inter-packet time        differences in network latency L1 352. This is not the        traditional calculation of jitter used for VoIP and other        real-time protocols. For those protocols they maintain an        expected arrival time for packets (e.g. every 20 ms), and jitter        is calculated as a filtered average of the differences between        actual and expected packet arrival times;    -   Received bandwidth;    -   Internal queue status.

The data rate of the stream may fluctuate as a function of theencoding/compression mechanism. Uncompressed frame data may berelatively constant, but may be too large to transmit reliably over theRF interface modules. Compressing may alleviate the size problem, but atthe cost of extreme size variability between the individual frames.Using an infinite GOP may decrease the variation in some cases. However,since a scene change may still require a new full frame, the variationmay not be entirely eliminated. Padding may be used to smooth the datarate and help ensure a consistent level of network service.

Padding may also be used to resend missing data packets, backlog data(from when a connection starts to decrease its data rate before thesource decreases), and when there is nothing else for example, the RFinterface modules may transmit duplicate packets.

An RF interface module may include transmitter hardware or may becoupled thereto. The transmission control and data reassembly 46 mayinclude receiver hardware or may be coupled thereto. In order tocalculate the one way (transmitter to receiver) latency, accurate timingsynchronization may be required. Other solutions may track and react tothe round trip latency. However, this may ignore a very basic propertyof the connected networks: network latency may be asymmetrical, with thenetwork latency from the transmitter to the receiver being more likelyto increase as the load increases. There typically is far more databeing sent from the transmitter than there is to the transmitter.Further complications may arise from wireless network configurations.Wireless networks may be constructed or configured to have a muchgreater proportion of the available bandwidth dedicated to download(sending to the transmitter) versus upload. Hence, the network latencymay be much more likely to increase on the upload/transmit side. Anysolution which depends on the round trip latency may understate thecurrent network latency and run in to transmission difficulties as aresult.

FIGS. 6 a and 6 b are schematic block diagrams of an exemplary system 80showing delivery of data packets using feedback mechanisms orconnectivity conditions. The embodiments discussed may extend to anenvironment where additional wireless networks are supported. FIGS. 6 aand 6 b also provide greater detail on the exchange of both controlinformation 114, 88 and 86 and Multiple Transport Buffers 16 between thecomponents of the system. Control information may refer to connectivityconditions or feedback. Feedback may also be sent from data transmissioncontrol and reassembly via link 44.

FIG. 6 b is a schematic block diagram corresponding to FIG. 6 a showingTransport Buffer 1 or TB1 (34) and Transport Buffer 2 or TB 2 (36) ingreater detail. As can be seen, TB 1 (34) contains odd numbered packets102, 104 and 106 and TB 2 (36) contains even numbered packets 108, 110and 112. Even though other Transport Buffers are not explicitly shown inFIG. 6 b for ease of illustration, it is understood that embodimentsshown in FIG. 6 b may comprise more than two Transport Buffers within orconnected/coupled to the Buffer Management and Transport Controller 28.In both FIGS. 6 a and 6 b, data Packets may be delivered via links 116to RF Interface Modules (or simply “RF Modules”), while feedbackinformation 114 (e.g. connectivity condition) may be provided to theBuffer Management and Transport Controller 28.

Within or connected/coupled to the Buffer Management and TransportController 28, each Transport Buffer TB-1 (34), TB-2 (36), TB-3 (38) . .. and TB-N (40) may act as a container for data packets (P), shown inTB-1 as P1 (102), P3 (104) and P5 (106) in some embodiments. If the databeing exchanged is audio and video data then these data packets may beindividual frame segments, portions thereof, or a collection of framesegments. If the data being exchanged is multimedia data then it couldalso be individual pictures, gestures or some other digital data, orportions thereof. Segmented data packets may be buffered in transportbuffers (e.g. as a long stream of data packets, or a set of datapackets) and may be re-assembled at the receiver. Data packets can beselectively and dynamically withdrawn from transport buffers anddelivered to a selected RF interface module (associated with one or morenetworks and one or more network carriers). The data packets arecontinuously and simultaneously transmitted across the RF interfacemodule and network for reassembly at the destination (e.g. receiver).

In the exemplary implementation, audio and video frame data placedwithin the transport buffer for sending is compressed, encoded, possiblyencrypted, and split into data packets for individual transport. Thedata packets may be sized such that when transmitted over theintervening networks (e.g. IP networks in some examples), they mayexperience no transparent fragmentation. The maximum size of a UDPpacket may be the determining factor in the number of data packetsrequired to represent an audio or video frame, for example.

The data packets make up a part of the original Data Streams (DS) 14coming from the Data Processor and Encoder 24 and Data Source 20. In theillustration shown in FIG. 6 b, the odd numbered packets 102, 104 and106 are contained by Transport Buffer 1 (TB-1) 34 and even numberedpackets 108, 110 and 112 are contained by Transport Buffer 2 (TB-2) 36.These may be sent respectively to RF Interface Module 1 (RF-1) 52 and RFInterface Module 2 (RF-N) 58. Other RF Interface Modules could bepresent, but for simplicity only two RF interface modules 52 and 58 areillustrated. Accordingly, data packets from the same original datastream 14 may be simultaneously transmitted using multiple interfacemodules 52, 58, and wireless networks 70, 72, 74. This may increasethroughput as transmission is not limited to a single interface module.Further, a single RF interface module 52 may connect to multiplewireless networks 70, 72, 74.

As Data Packets are delivered via links 116 to RF Interface Modules 52and 54 respectively, feedback information 114 (e.g. connectivitycondition) is provided to the Buffer Management and Transport Controller28 from RF-1 (52) and RF-2 (54). This information includes, for example,Network Latency, lost packet reports, RF Interface Module bufferfullness, RF metrics (e.g. RSSI, RSCP, EC/IO, CQI) and their rates ofchange, time between successful transmission attempts, current availablebandwidth, historical information tabulated from previous readings, andso on. This feedback information assists the Buffer Management andTransport Controller 28 to determine the best means or route fordelivering additional data packets to follow. The selection of an RFinterface module is made before the additional data packets aretransmitted. The selection of each subsequent Data Packet to be given toall available RF Interface Modules 52 and 54 is made based on as muchfeedback information 114 as can be gathered by the RF Interface Modules52 and 54 regarding various components of the system. In this system theability to feedback relevant status information (e.g. connectivitycondition) about current connected network conditions to the BufferManagement and Transport Controller 28 may improve the overallperformance of the system, may optimize use of RF interface modules, andmay increase throughput.

Additional options may also be possible in an exemplary system to dealwith feedback 114 from RF Interface Modules 52 and 54. For example ifthe calculated aggregate throughput (e.g. aggregate of individualthroughput) based on the feedback 114 is less than the current sourcedata rate (coming from the Data Processor and Encoder) and/or there is asystemic inability to deliver data packets to the receiver then theBuffer Management and Transport Controller could send a throttle command88 to the Data Processor and Encoder 24. In this exemplary embodimentthe Data Processor and Encoder 24 may have additional functionalityadded to deal with any feedback 88 given. This feedback might haveseveral effects, such as limiting the number of Data Streams 14 sent tothe Buffer Management and Transport Controller 28, or reducing theoutput resolution, quality and/or frame rate. This may also involveincreasing or decreasing the compression rate and characteristics. Withfuture advanced Data Source equipment, such as high-end productioncameras, the Data Processor and Encoder 24 might have the option ofsending a control signal 86 to a more advanced Data Source 20 to reducethe quality and quantity of information being sent. In the example of aproduction camera this advanced Data Source 20 might be able to changethe picture resolution dynamically, or perhaps inform the user of theData Source 20 equipment that a change in setting is required. Thismeasured feedback mechanism through the Buffer Management and TransportController 28 may be programmatically determined based on parameterssuch as buffer sizes, high water and low water thresholds and otheralgorithms.

FIG. 7 is a block diagram of another exemplary system 130 showingdelivery of data packets over multiple wireless networks 70 and 72 usingfeedback mechanisms. In this example RF Interface Modules 52, 54 mayconnect or couple to the same wireless network 70, and RF InterfaceModule 134 may connect to an additional wireless network 72. Theselection of two wireless networks and three RF Interface Modules is maybe exemplary and non-limiting. Other examples may include a greaternumber than shown in this illustration.

In FIG. 4 the Buffer Management and Transport Controller 28 hasutilities to use the additional RF Interface Module 134 to furtherimprove the reliability of delivering data packets. As feedback 114, isreceived from all RF Interface Modules 52, 54 and 134 the BufferManagement and Transport Controller 28 determines which wireless links136 and wireless networks 70 and 72 are performing well or poorly. Giventhe wide range of encoding methods available to the Buffer Managementand Transport Controller 28 it is possible to prioritize some DataPackets (102, 104, 112 and 138) as more important than others. Thesemore important Data packets (102, 104, 112 and 138) could be consideredessential to reassembling the data. In the example of audio and videodata being encoded, these data packets may be considered essential tomaintaining usable picture quality, while other data packets may beconsidered incremental to improving picture quality (e.g. highdefinition or high framerate) components and are therefore lessessential.

In marginal circumstances (where the bandwidth is available for bothvideo and audio, but it is either error prone or close to the cut-offfor viable video), the Buffer Management and Transport Controller 28 maychoose to only transmit the audio stream, or to transmit a reduced framerate video stream. In such a scenario, the receiver displays a delayedor frozen picture while audio continues normally.

When the transmitter goes into audio-only mode, video can be stillencoded but not transmitted. Audio is prioritized and duplicated(whenever possible) across all available connections.

Another embodiment enhances audio-only mode with a low-framerate,non-real time video stream. Audio is encoded and transmitted inreal-time, but video is encoded at a significantly reduced frame rate—1frame every 3 seconds, or similar. The resulting video frames are markedas not real time and sent to the receiver. This allows the receiver toaugment the audio stream with a periodically updating “slideshow” whenthe circumstances of the connected networks do not allow for a completevideo stream.

At all other times, the Buffer Management and Transport Controller 28may use the space reserved for padding to duplicate important frames(typically video key frames and audio frames) to minimize thepossibility of display error and improve output recovery times whendisplay errors do occur.

In this example the Buffer Management and Transport Controller 28 mayreplicate these Data Packets (102, 104, 112 and 138) within TransportBuffer 3 (TB-3) 38 and may send them to RF-3 (RF interface modules 134)for wireless transmission. In the exemplary implementation, higherpriority packets may also be transmitted as duplicates in the paddingcomponent. Alternate implementations may also transmit higher prioritypackets on RF interfaces judged to be the most reliable and/or fastest,with the lowest network latency. In the event that the Data TransmissionControl and Re-assembly module 46 receives duplicate frames a mechanismmay detect and delete the duplicates. By adding additional logic andintelligence within the Data Transmission Control and Re-assembly module46, additional links to wireless networks and duplicate detection can beadded. If the essential Data Packets within Transport Buffer 1 (34) andTransport Buffer 2 (36) are lost in transmission then the redundanttransmission of those Data Packets within Transport Buffer 3 (38) mayensure their arrival. As noted, Data Transmission Control andRe-assembly module 46 may provide feedback to Buffer Management andTransport Controller 28 for use in routing decisions.

In the exemplary implementation, the receiver (e.g. Data TransmissionControl and Re-assembly module 46) may notify the sender that aparticular data packet never arrived. The Buffer Management andTransport Controller may then select on one of the other, alternate RFinterface modules 52, 54, 134 to resend it. Transmission of lost datapackets may be prioritized over regular stream data. Transmission oflost data packets may also replace some or all of the bandwidth reservedfor padding. Ideally, the fastest available connection resends themissing data packet to maximize the time available for frame reassembly.

FIG. 7 illustrates a reassembled data stream 22, wherein datatransmission control and re-assembly 46 reassembles data packets (P1 toP8) 22 using timing and sequence information.

FIGS. 6 a, 6 b and 7 provide example illustrations of how a singlecontinuous data stream 22 may be segmented into multiple data packets(P1-P8), transmitted simultaneously or concurrently over differentwireless networks using different RF interface modules, and re-assemblyat the receiver side. As noted herein, this may aggregate datathroughput from different wireless networks. Further, routing on aper-packet basis based on continuous feedback may optimize usage ofwireless networks and RF interface modules. One data packet may go toone network and another data packet may go to another (or different)network. This is different than sending the same data packets overdifferent network paths (of the same network), or sending the same datapackets over different networks, where selection may be made after thedata packet has been transmitted. Instead, embodiments described hereinmay select an estimated optimal (or near optimal) RF interface moduleand network prior to transmission of the data packet.

FIG. 8 is a block diagram of an exemplary system showing additionalfeedback mechanisms (e.g. connectivity conditions). This simplifiedexample illustrates one wireless network 70 for simplicity, but multiplewireless networks 70, 72, 74 may form part of system 200. The feedback114 from RF Interface Modules 52 and 54 may still be available, butaugmented with feedback information 202 coming from the DataTransmission Control and Re-assembly 46 component (e.g. receiver). Thisadditional feedback 202 may be used to carry information aboutsuccessful delivery of Data Packets, information about how long it tookeach data packet to arrive (i.e. network latency) and many othercharacteristics about the reception of data packets. The BufferManagement and Transport Controller 28 processes the received feedbackfrom the RF Interfaces and the receiver. Based on the calculated healthof the transmission, the Buffer Management and Transport Controller 28instructs the Data Processor and Encoder 24 to increase or decrease thecurrent data rate. The Data Processor and Encoder 24 may instruct theData Source 20 to limit the quantity and quality of information beingsent.

In a wireless environment feedback from a range of sources may improvecomplex logic made by controller when sending large amounts of timecritical digital information. The Buffer Management and TransportController 28 may improve its success rate of data packets deliveredwith each new piece of information provided to it, and may also improvetransmission rate by making decisions based on continuous updatedfeedback.

Feedback may relate to information received from the application layer(layer 7 from OSI model, for example). The application layer may be theclosest layer to the end user, such that both the application layer andthe user may interact directly with the software application. Theapplication layer may interact with software applications of embodimentsdescribed herein that implement a communicating component and controlunderlying hardware components. An application layer implementation maynot require knowledge of underlying network operations.

A data packet in accordance with embodiments described herein may bedifferent from an UDP/IP packet. Ideally a data packet is sizedappropriately to fit with a UDP/IP packet without further fragmentation,although, in some circumstances, on some connections, this may not bepossible. An advanced embodiment may detect the maximum UDP/IP packetsize and size data packets appropriately.

The application layer formats a data packet into one or more UDP/IPpacket(s) and delivers the result to the RF Interface for delivery tothe Internet.

In implementations where the RF Interface is replaced by an Ethernetconnection, no further fragmentation is required.

For RF interfaces connected to a cellular network, the UDP/IP packet maybe further fragmented by the radio layer for delivery across thewireless network to the base station. This network layer is responsiblefor fragmenting the UDP/IP packet (into pieces) according to theconnected network proprietary protocols and requirements. Radiotransmission of data to (and from) the base station may use technologiesthat are not contemplated in this document.

The network layer within the connected network reassembles the UDP/IPpacket before forwarding it to the Internet for delivery to thereceiver.

Some implementations may involve an iterative process of improvement inlistening to and adapting to feedback from the RF interfaces, thewireless networks, and the receiver (e.g. data transmission control andre-assembly) in the studio. Various feedback loops are shown in thedrawings, including feedback 202 from data transmission control andre-assembly 46, feedback 114 from RF interface modules 52, 54, 56, 58,feedback 88, 406, 404 from buffer management and transport controller28, and feedback 86 from data processor and encoder 24.

An example implementation may consider RF interface buffer fullness,measured one-way network latency, and lost packets (the difference involume between transmitted and received packets) as a mechanism toidentify whether a specific connection (carrier network and/or RFinterface module) is working well. If the RF interface buffer fills up,it indicates that there is resource contention or interference at the RFlayer, so the transmit data rate may be adjusted downwards tocompensate. Similarly, if the one-way network latency increases, itmeans increased buffering by some network element between thetransmitter and receiver, which will eventually result in packet loss ifthat buffer becomes full—in either of these scenarios, reducing thetransmit data rate is prudent to maintain real-time system performance.When the RF interface buffer empties, the one-way network latency drops,or packet loss is reduced, the transmit data rate may increase. Thisexample implementation may lead to a situation where the transmitter mayramp up the data rate on a connection until data packets either start tobecome effectively lost by the network, or the one way network latencyincreases to a point where it is no longer usable for the selectedglass-to-glass latency.

Lost packets are typically an indicator of network congestion, but in awireless environment loss can also occur randomly due to physicalinterference. As such, this example embodiment places less weight onpacket loss as a metric, preferring RF interface buffer fill and one-waynetwork latency as earlier and more reliable indicators of congestion.Future embodiments that are able to reliably discriminate between lossdue to congestion and random loss due to wireless effects can put aheavier weight on loss as a metric for determining transmit data rate.

As a further example, embodiments described herein may involvecharacterization of the channel, which may be used as part of feedback.Buffer management and transport controller may allocate data to aparticular channel based on the characterization.

Another implementation may consider both the transmitted bandwidth andthe received bandwidth. Comparing the two values may give anapproximation of the amount of data actually traversing the connectednetwork, and how much of a backlog is building up. In the ideal world,these two values may be equal (transmitted==received). In practice,however, this may not be possible as there may typically be a delaybetween when a packet is transmitted and when it is received, e.g. thenetwork latency. Without some mechanism to account for this delay andalign the bandwidth reports, there may be no mechanism to accuratelydetermine the actual health of the network.

As a result, another implementation may involve accurate timesynchronization between the transmitter and the receiver. The timesynchronization may be referred to as feedback. An embodiment may makeuse of the time synchronization to determine the current networklatency. The network latency, as defined previously, is important tomodel the behaviours of the connected network. Note that the networklatency is not the same as measuring the round trip time (frequentlydenoted RTT) from the transmitter to the receiver and back to thetransmitter again and dividing it in half. There is no guarantee that ½RTT is the same as the network latency since the time for data totraverse the connected network may not be symmetrical. Due to the designand nature of some connected networks the time required for a datapacket to travel from the transmitter to the receiver (upload) may bemuch longer than the time for the same packet to travel from thereceiver to the transmitter (download). If they are not the same, andthe algorithm used ½ RTT, inaccurate timing may result.

Before an embodiment starts a data stream, the transmitter and receivermay synchronize their clocks. The transmitter sends a data packet to thereceiver with the transmitter's current time. When this packet arrivesat the receiver, the receiver's time stamp is added the packet is sentback to the transmitter. The packet is passed back and forth a secondtime, again collecting time stamps at each stop. Once the packet has twopairs of time stamps (minimally after two round trips), the transmitteris able to calculate an initial time correction offset. The offset isstored, and the packet stamped and sent again on another round trip.This allows either the transmitter or the receiver to calculate theround trip latency. The mechanism is repeated until the round triplatency comes to a minimum (averaged over a period, looking for a bestcase). This value is then halved to approximate the one-way latency.Based on the information in the time packet, the transmitter is able todetermine the difference between the transmitter and receiver clock andcalculate an offset for conversion. A final time packet is sent to thereceiver to confirm the time offset and allow the receiver tosynchronize its clock to the transmitter.

This process must be repeated periodically (e.g. every five minutes) toaccount for clock drift—tiny variations in CPU frequencies that causethe time offset between the transmitter and the receiver to shift overtime. If the time synchronization is not updated, the resulting errormay cause the transmitted stream to fail.

Typically, this requires the process to be repeated during an activebroadcast (while the transmitter is actively sending a live videostream) over a potentially heavily loaded channel. It is not possible tohalt the stream (even over one of the available channels) or reserve achannel to dedicate for time synchronization. The algorithm accounts forthe potential asynchronous nature by sending the time packet over everyavailable channel and selecting the best case—the first one delivered.This, combined with the repetition, allows the algorithm to come to astable minimum and determine an accurate offset.

In some cases, clock drift between the transmitter and receiver may beso severe that the period of time synchronization (5 minutes orotherwise) may not be frequent enough to result in a usable value forthe measured distance between the transmitter and receiver clocks. Theexample embodiment compensates for this scenario by calculating anaverage velocity for the measured distance. Each time thesynchronization algorithm completes, an additional data point fordistance is recorded—when there are at least two distance measurements,a least squares trend line for velocity can be calculated, which is thenused to approximate the current distance until the next time thesynchronization algorithm is run.

Once an embodiment has synchronized the time between the transmitter andthe receiver, determining the network latency, on a per data packetbasis, is straightforward. Each data packet may be time stamped with thecurrent time at the transmitter before sending. Once the packet arrivesat the receiver, the time stamp is adjusted using the calculated offset,compared against the current time, and the network latency determined bya simple mathematical calculation.

Time synchronization may allow the transmitter to align bandwidthreports with the receiver and know exactly (within the resolution of thesystem clocks) when a data packet has been received. An implementationwith timing may simply align transmitted segments with receivedsegments. While workable, it may be complicated, particularly when adata packet is delayed enough to push it into the next time segment.

Feedback information with respect to network health may consider one-waynetwork latency. That is, one-way network latency may be an example offeedback as used herein for routing decisions. As the load on a networkincreases, internal queues (to the connected network) may start to growand data packets may be delayed. As this is a delayed real-time system,the delay is the important piece of information. Packets delayed toolong might as well be lost as their parent frame may need to bereconstructed on time or fail. By monitoring the one-way delay from thetransmitter to the receiver (e.g. the network latency), and watchingboth the absolute value and the rate of change in the value of one-waynetwork latency, the transmitter or controller is able to anticipatewhen an RF interface module may be overloaded and react before failureoccurs. Accordingly, embodiments described herein may estimate one-waynetwork latency which may be used by system as feedback. The one-waynetwork latency may enable embodiments described herein to account forasynchronous load on a network.

Lost packets may be good for indicating a connection has failed. RSSImay be good for indicating a connection might be usable, but gives noindication of actual network loading. Network latency may allow thesystem to anticipate connection failure, and re-route data packetsaccordingly.

FIG. 9 is a block diagram of an exemplary system 400 showing multipleoccurrences of buffer management and transport controllers working overmultiple wireless networks such as 460 and 462 with a central controller402 providing additional feedback mechanisms. In a system with multiplewireless networks such as 460 and 462, and multiple occurrences of thesystem being used, the system allows for an additional feedback 404mechanism to further assist with the delivery of Transport Buffers suchas 444, 446 and 448. Due to the volume of information that must betransmitted from Buffer Management and Transport Controllers 412, 430and 432 it must be recognized that serious loads may be placed upon oneor more Wireless Networks 460 and 462. Although five base stations areshown over two wireless networks, these are limitations of theillustration and do not represent real limitations. The nature of largequantities of data such as normal definition and high definition audioand video may require advanced methods to be undertaken to ensuresuccess in the overall system 400. As the number of installed andworking systems in a given region increase they will require a CentralBuffer Control and Base Station Distribution 402 component to assist inimproving the overall system 400.

In system 400 as Buffer Management and Transport Controller #1, #2 and#N (412, 430 and 432) receive feedback information 114 and 202 it isacted upon, but also routed as feedback information 406 to the CentralBuffer Control and Base Station Distribution 402 component. If feedbackinformation 202 is not present then only 114 is used from each RFInterface Module 444, 446 and 448. The ability to couple the BufferManagement and Transport Controllers 28, 430 and 432 to the CentralBuffer Control and Base Station Distribution 402 component is managedthrough a wide-area Network A 410. This Network A may be a well-knownnetwork such as the Internet or it could be a private IP network.Connection into this network could be wired or wireless, for example aWiFi access point running in conjunction with the Internet or some othercombination.

The resulting effect of having the Central Buffer Control and BaseStation Distribution 402 component overseeing the system allows actionsto be taken between different Buffer Management and TransportControllers 412, 430 and 432. For example the Central Buffer Control andBase Station Distribution 402 component may detect that network 70 isreaching a saturation point and request that RF Interface Module 446move its connection to another network 72 that is underutilized. Thesekinds of oversight decisions allow the system to be better balanced andallow it to provide additional control in the system to improve thethroughput of Transport Buffers 16.

FIG. 10 provides a flow chart illustrating the flow of data from themobile data source to the destination receiver. The data information isreceived from a data source and encoded into the selected type of datastream at step 502. When the data source is a video source it could behigh definition (HD) broadcast quality data from a digital camera, someother direct source, or stored information that has originally createdfrom a direct source. Other sources could include multimedia data,logging data and any form of data that is produced in large quantitiesthat has a time critical aspect to the information. Depending on thedata type, it can be encoded using many techniques to reduce the volumewherever possible. In the case where a video stream is being encoded, itis possible to follow one of many encoding methods that includecompression, such as MPEG-4 and Multiple Descriptive Coding (MDC)discussed earlier.

This data stream may be encoded into data packets and stored intransport buffers (TBs) at step 504, based on a variety of parametersfor eventual transmission. Some of these parameters include the numberof available RF Interface Modules for transmission, the feedback fromeach of the RF Interface Module, the type of redundancy that is requiredfor the data based on the number of available RF Interface Modules.Storage of data packets in transport buffers provides some lag time inthe arrival of the data packets and subsequent frame reconstruction, butallows for dynamically changing wireless delivery conditions to be dealtwith more effectively.

The feedback information from all sources is interpreted by controller(e.g. Buffer Management and Transport Controller 28) to make RFInterface Module routing choices at step 506. Feedback information mayinclude network latency, RSSI, dramatic RSSI changes, sender bandwidth,receiver bandwidth, calculated throughput based on a time interval,instant throughput with delay at receiver versus sender, highestbandwidth sent per link, and average packet loss per link.

The interpretation of the feedback can result in a determination ofwhether one or more attached RF Interface Modules are being given toomuch data to transmit at step 508. If the data is not a problem a checkis performed at step 512 to determine if a previous command was sent toreduce the amount of data being sent at step 512. If the amount of datahas been restricted and the restriction is no longer required a commandis sent to the Data Processor and Encoder 24 and possibly the DataSource 20 to increase the amount of data being processed, at step 518.

If one or more RF Interface Modules are receiving too much data then acommand may be sent to the Data Processor and Encoder 24 at step 510 toreduce the amount of Data Stream information. A check may then be madeat step 514 to see if the data source 20 can receive a command to reduceinformation. If it can, then a command is sent to the data source 20 atstep 516 to reduce the quality and quantity of information to beencoded.

After these adjustments are made to the system a data packet from aTransport Buffer is pulled from memory and sent to the correct RFInterface Module based on the programmed logic, the feedback that hasbeen processed and any parameters that have been set by the user at step520. Parameters could include high and low watermarks for each attachedRF Interface Module, the maximum saturation of transmitted data througha given Wireless Network based upon total transport buffers transmitted.

In some examples, video encoders can have a higher latency—for example,at normal GTG values, the latency for video encoders can be typically inthe 500-700 ms range. For sub-second GTG this is a significant portionof the available network buffer. To help counteract this the amount oflookahead can be reduced at the expense of reduced compressionefficiency (i.e. more bits are required for the same video quality).

In other examples, switching away from a normal GOP with large 1-framesto a mode (periodic intra refresh) where 1-macroblocks are spread overmultiple frames is possible in order to support low G2G latency. This isbecause a typical 1-frame might be anywhere from 10-30× larger than aP-frame.

For example, if the encoder is configured for 1 Mbit/s (e.g. since theaggregate connection throughput is 1 Mbit/s), and the initial I-framethat comes out of the encoder is 500 Kbit. That I-frame alone will take500 ms to arrive at the player, again significantly eating into asub-second GTG. Spreading intra-information over multiple frames allowsthe frames to be more uniform in size, which can prevent this problem.FIG. 11 is a flow chart illustrating the flow of data to determine whichpaths to use for delivery it to a destination. This more detailed flowchart expands the logic within steps 506 and 508 of FIG. 10.

At step 602 there are many elements of state information that can beexamined to determine the best course of action (e.g. selection of RFinterface module and/or network based on the feedback). The examinationsthat follow at steps 604, 610, 620 and 630 are a subset to represent thetypes of detailed behaviors that can be examined for. This stateinformation may include internal state information (such as the size ofcurrent buffer levels), external state information (such as feedbackfrom the RF Interface Modules), and Central Buffer Control Information(such as the overall saturation of a given base station across alloccurrences of the invention).

Examination step 604 examines the current RSSI value from a given RFInterface Module. The RSSI value can be used in many ways. Astraightforward high water and low water mark could be used to determineat this point in time if an RF Interface Module is maintaining goodcoverage with the currently attached base station. A historical metriccould be used to determine if the RSSI value has changed toodramatically from the last RSSI value provided by an RF InterfaceModule. Several RSSI levels may be averaged to determine if there is anegative trend that indicates a pending failure. Whatever technique isused the RSSI may be a very useful feedback element from the RFInterface Module to determine what is happening if the RF interfacemodule link to a given Wireless Network.

At step 606, if the RSSI value is a concern then the amount of TransportBuffer information given to this RF Interface Module is reduced. A flagis then set at step 608 to indicate the RSSI value for this RF InterfaceModule is a concern and processing moves to step 610.

If the RSSI value was not a concern, or after setting an RSSI concernflag, a check is performed on the current transmit buffer level at step610. This internal value could be used to indicate that the buffer forthis RF Interface Module is being drained too slowly compared to apreconfigured rate of usage. This rate could also be seen as the rate atwhich the transport buffer is being emptied (e.g. output rate) comparedto how quickly it is being filled (input rate). If this transport bufferexhaustion rate falls below a determined value then the number oftransport buffer being given to a specific RF Interface Module isreduced at step 612. The buffer level bad flag is set for this RFInterface Module at step 614 and the processing proceeds to step 620.Embodiments described herein may match the rate of data going into atransport buffer (input rate) with the rate of data taken out of thetransport buffer (output rate) based on received feedback of networkconditions (both transmitter and receiver related conditions). The inputrate may be responsive to the output rate, as determined based on thefeedback of network conditions.

If the buffer rate was not a concern, or after setting the buffer ratewith a bad flag, a check is performed on the packet loss per connectionat step 620. This calculated value may be performed in several ways. Forexample, the number of transmit attempts through a given RF InterfaceModule may be compared to a success rate. The number of base stationcollisions when transmitting could be compared to the number oftransmits attempted. When too many collisions occur on an average amountof transmit attempts it usually indicates that the base station isgetting saturated and is not keeping up. If the packet loss per RFInterface Module connection is too high then the number of transportbuffers given to this RF Interface Module is reduced at step 622. Apacket loss problem flag is set and processing moves to step 60.

Other possible checks include the total number of active connections andchanges to those connections. Looking at the uptime per connection andcomparing that to the amount of time the RF Interface Module is actuallyconsidered out of coverage and number to communicate with the basestation. The throughput per connection based on feedback from thedestination receiver could also be examined. This feedback could alsohave the time delay or latency for transmitting a transport bufferthrough a given wireless network. Current state information could alsoinclude GPS location data, and the time of day. Time of day informationcould allow for checks on peek and busy periods of the day, allowingcertain RF Interface Modules to avoid heavier transmissions in certaindense downtown regions during peak periods. Overall sustainablebandwidth can also be determined based on data transmit per RF InterfaceModule averaged per second or minute.

Once all possible state informational elements are examined, an overallcount of the number of flags set is performed at step 640. This checkcould be done on one RF Interface Module, or across all RF InterfaceModules. Some programmed or configured high-water mark might have to bereached in order to take the bigger step of reducing the amount of datato be encoded into transport buffers at the source. If the number offlags set was too high, then processing moves to step 642, where a flagis set to reduce data at the source. Processing then moves to step 510of FIG. 10. The reduce data at source flag will be used later when itmight be necessary to turn on full data flow once again. If the numberof flags set were not too high then processing proceeds to 512 of FIG.10, where the flag will be checked and the amount of data being set maybe increased.

Although embodiments have been described herein with reference tocertain specific embodiments, various modifications thereof will beapparent to those skilled in the relevant art.

We claim:
 1. A system for transmitting large volumes of source data overa plurality of wireless networks comprising: a buffer management andtransport controller configured to divide an encoded data stream into aplurality of data packets for buffering into multiple transport buffers;and a plurality of radio frequency interface modules configured towirelessly transmit the plurality of data packets to the plurality ofwireless networks to aggregate throughput across the plurality ofwireless networks, wherein the plurality of radio frequency modules arefurther configured to provide wireless connectivity conditions to thebuffer management and transport controller regarding the plurality ofwireless networks, wherein the plurality of wireless networks areoperated by two or more network carriers; wherein the buffer managementand transport controller is further configured to process the wirelessconnectivity condition to: for a data packet of the plurality of datapackets and prior to a transmission of the data packet, withdraw thedata packet from a transport buffer of the multiple transport buffersand select a radio frequency module for the transmission of the datapacket over at least one of the plurality of wireless networks.
 2. Thesystem of claim 1, further comprising a data transmission control andreassembly module configured to receive the plurality of data packetsfrom the plurality of wireless networks for reassembly.
 3. The system ofclaim 2, wherein the data transmission control and reassembly module isconfigured to provide receiver feedback to the buffer management andtransport controller regarding additional wireless connectivityconditions for the plurality of wireless networks, and wherein thebuffer management and transport controller is further configured toprocess the receiver feedback to select the radio frequency module forthe transmission of the data packet.
 4. The system of claim 1, whereinthe buffer management and transport controller is configured to match aninput rate for the plurality of data packets with a withdraw rate forthe plurality of data packets at a transport buffer of the multipletransport buffers using the wireless connectivity conditions.
 5. Thesystem of claim 1, wherein the wireless connectivity condition comprisesone-way network latency, and wherein the one-way network latencycomprises a delay between when a data packet is transmitted and when thedata packet is received.
 6. The system of claim 1, wherein at least oneradio frequency interface modules of the plurality of radio frequencyinterface modules concurrently maintains multiple network connections tothe plurality of wireless networks.
 7. The system of claim 1, whereinthe transport controller is further configured, using the receivedwireless connectivity conditions, to dynamically and selectivelyredirect a data packet of the plurality of data packets from one radiofrequency interface module to another radio frequency interface moduleof the plurality of radio frequency interface modules, and from oneavailable wireless network to another available wireless network of theplurality of wireless networks, whereby data throughput to a destinationreceiver is maintained.
 8. The system of claim 1, wherein the sourcedata is transmitted in real-time with a constant delay.
 9. The system ofclaim 1, further comprising a data source configured to capture acontinuous real-time stream of audio-video data for the encoded datastream.
 10. The system of claim 1, wherein the wireless connectivityconditions comprise information relating to: an error rate for each ofthe plurality of radio frequency interface modules based on theassembled packets that are not delivered at the destination forre-assembly; and a capacity condition for each of the plurality of radiofrequency interface modules based on the assembled packets that aredelivered at the destination for re-assembly.
 11. The system of claim 1,wherein the buffer management and transport controller is configured tocontrol: delivery of each data packet of the plurality of data packetsfrom a transport buffer of the multiple transport buffers to acorresponding radio frequency interface module of the plurality of radiofrequency interface modules; and transmission, for re-assembly at adestination, of each data packet of the plurality of data packets fromthe plurality of radio frequency interface modules for wirelesstransmission over one of the plurality of wireless networks.
 12. Thesystem of claim 1, wherein the buffer management and transportcontroller is configured to use the received wireless connectivityconditions to control at least one of the quantity and the quality ofthe captured continuous real-time stream.
 13. The system of claim 1,wherein the buffer management and transport controller is furtherconfigured to route redundant packets to the plurality of radiofrequency modules for increased transmission reliability.
 14. The systemof claim 1, wherein the wireless connectivity conditions are selectedfrom the group consisting of: one-way network latency, glass-to-glasslatency, transmit collisions at one or more said simultaneouslyavailable different wireless networks; the number of the modules in thewireless radio transmission with each said simultaneously availabledifferent wireless network; throughput requirements of the plurality ofmodules for the wireless radio transmission on each said simultaneouslyavailable different wireless network; coverage drop-off indicators; timebetween successful transmission attempts; status values; Received SignalStrength Indicator (RSSI); packet retransmissions; average packet lossper link; sender bandwidth; destination bandwidth; current availablebandwidth; and combinations thereof.
 15. The system of claim 1, whereinthe buffer management and transport controller is further configured,using the wireless connectivity conditions, to control a transmissionrate of data packets to a corresponding radio frequency interface moduleof the plurality of radio frequency interface modules from the multipletransport buffers.
 16. The system of claim 1, wherein at least one ofthe plurality of radio frequency interface modules is configured toapply congestion control to establish and maintain a transmission rateof data packets.
 17. The system of claim 16, wherein the at least one ofthe plurality of radio frequency interface modules is configured toapply congestion control by determining a congestion avoidancethreshold.
 18. The system of claim 17, wherein an initial value of thecongestion avoidance threshold is determined as a function of a measuredtransmission rate.
 19. The system of claim 18, wherein a targettransmission rate is determined based on at least one of the congestionavoidance threshold and a measured transmission latency.
 20. The systemof claim 18, wherein the congestion avoidance threshold is increasedwhen a measured transmission latency is within a pre-defined range. 21.The system of claim 19, wherein the at least one of the plurality ofradio frequency interface modules is configured to transmit data packetsat the target transmission rate.
 22. The system of claim 1, wherein theplurality of radio frequency interface modules are configured totransmit the plurality of data packets to the plurality of wirelessnetworks simultaneously or near simultaneously.
 23. The system of claim1, wherein each of the plurality of wireless networks have disparatenetwork characteristics or conditions.
 24. A method for transmittinglarge volumes of source data over a plurality of wireless networks, themethod comprising: dividing an encoded data stream into a plurality ofdata packets; buffering the plurality of data packets into multipletransport buffers; determining wireless connectivity conditionsregarding the plurality of wireless networks, wherein the plurality ofwireless networks are operated by two or more network carriers;wirelessly transmitting the plurality of data packets to the pluralityof wireless networks to aggregate throughput across the plurality ofwireless networks; and processing the wireless connectivity conditionto: for a data packet of the plurality of data packets and prior to atransmission of the data packet, withdraw the data packet from atransport buffer of the multiple transport buffers and dynamicallyselect a radio frequency module for the transmission of the data packetover at least one of the plurality of wireless networks.
 25. The methodof claim 24, further comprising receiving the plurality of data packetsfrom the plurality of wireless networks for reassembly.
 26. The methodof claim 24, further comprising receiving receiver feedback regardingadditional wireless connectivity conditions for the plurality ofwireless networks, and further processing the receiver feedback toselect the radio frequency module for the transmission of the datapacket.
 27. The method of claim 24, wherein the wireless transmittingcomprises concurrently maintaining network connections to the pluralityof wireless networks.
 28. The method of claim 24, wherein the pluralityof data packets are transmitted such that source data is transmitted inreal-time with a constant delay.
 29. The method of claim 24, furthercomprising capturing a continuous real-time stream of audio-video data,and encoding the captured stream of audio-video data to form the encodeddata stream.
 30. The method of claim 24, wherein said wirelesslytransmitting comprises applying congestion control to establish andmaintain a transmission rate of data packets.